Towards Communicative Face Occlusions: Machine Detection of Hand-over-Face Gestures

Emotional body language constitutes an important channel of nonverbal information. Of this large set, hand-over-face gestures are treated as noise because they occlude facial expressions. In this paper, we propose an alternative facial processing framework where face occlusions instead of being removed, are detected, localized and eventually classified into communicative gestures. We present a video corpus of hand-over-face gestures and describe a multi-stage methodology for detecting and localizing these gestures. For pre-processing, we show that force fields form a better representation of images compared to edge detectors. For feature extraction, detection and localization, we show that Local Binary Patterns outperform Gabor filters in accuracy and speed. Our methodology yields an average detection rate of 97%, is robust to changes in facial expressions, hand shapes, and limited head motion, and preliminary testing with spontaneous videos suggests that it may generalize successfully to naturally evoked videos.